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EXECUTIVE  SUMMARY 


The  major  technical  accomplishments  achieved  during  this  grant  were 

i)  the  introduction  and  development  of  the  generalized  hill  climbing  algorithm  paradigm  as  a  unifying 

approach  to  address  intractable  discrete  optimization  problems, 

ii)  the  description  of  numerous  new  search  strategies  for  intractable  discrete  optimization  problems 

using  the  generalized  hill  climbing  algorithm  framework, 
in)  the  formulation  of  sufficient  convergence  conditions  for  generalized  hill  climbing  algorithms. 
iv)  the  application  of  several  generalized  hill  climbing  algorithms  to  an  integrated  blade  rotor 
manufacturing  design  optimization  problem. 

In  addition,  several  other  problems  were  studied  during  the  grant  period,  including  estimation 
procedures  for  discrete  event  simulation  problems,  access  control  security  system  issues,  and  flexible 
assembly  design  and  scheduling  problems. 

All  the  accomplishments  are  documented  in  several  archival  journal  articles  and  conference 
proceedings.  In  addition,  many  of  the  results  have  been  presented  at  national  and  international  conferences, 
and  have  won  awards  for  their  contribution. 

One  Ph.D.  dissertation  was  completed  during  the  period  of  the  grant.  Major  Alan  W.  Johnson, 
USAF,  successfully  defended  his  Ph.D.  dissertation  "Generalized  Hill  Climbing  Algorithms  for  Discrete 
Optimization  Problems"  in  October  1996.  Major  Johnson  also  received  an  honorable  mention  prize  in  the 
1997  George  E.  Nicholson  Student  Paper  Competition,  a  national  competition  sponsored  by  the  Institute 
for  Operations  Research  and  the  Management  Sciences  (INFORMS). 
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1.  Generalized  Hill  Climbing  Algorithms 

Generalized  hill  climbing  algorithms  provide  a  unifying  framework  to  address  intractable  discrete 
optimization  problems.  To  formally  introduce  and  describe  generalized  hill  climbing  algorithms,  several 
definitions  are  needed. 

Define  the  solution  space,  Cl,  to  be  the  set  of  all  solution  for  a  discrete  optimization  problem. 
Define  an  objective  Junction  f:  Q  ->R  that  assigns  a  real  value  to  each  element  of  the  solution  space. 

Define  a  neighborhood  structure  rr.Cl— >2^,  where  n(aJc:Cl  for  all  ©  eQ.  The  neighborhood 
structure  provides  connections  between  solutions  in  the  solution  space,  hence  allows  the  solution  space  to 
be  traversed  or  searched  by  moving  between  solutions.  The  goal  is  to  identify  the  globally  optimal  solution 
<0  *  (i.e.,  f(to  *)<f(©)  for  all  03  eQ),  or  more  realistically,  a  near-optimal  solution. 

1.1  Algorithm  Description 

The  components  of  generalized  hill  climbing  algorithms  are  summarized  in  pseudo-code  form. 

Select  an  initial  solution  ©q  efi 
Define  an  objective  function  f:  fi  — 

Define  a  neighborhood  structure  n:  Cl— >  2^ 

Set  the  iteration  index  k=0 

Define  the  hill  climbing  (random)  variables  Rk  :  ChiCl  ->Uu  {-oo,+oo} 

Repeat 

Generate  a  solution  03 '  e  n( to  k)  in  the  neighborhood  of©  k 
Set  A(©  k  ,03 ')  =  f(© ')  -  f(to  k) 

If  A(©  k  ,03 ' ) <  0,  then  ©  k+l  ® '  (objective  function  decreases) 

If  A(©k,©')^  0  Rk(©k>®')^A(©k,©')> 

then  ©  k+l  ® '  (accept  hill  climbing  move) 

If  A(©k>©')-  0  Rk(®k>®0  <^(£0k»a),)> 

then  ©  k+i  <-  ®  k  (reject  hill  climbing  move) 

k<-k+l 

Until  stopping  criterion  is  met 

Generalized  hill  climbing  algorithms  allow  inferior  solutions  to  be  visited  enroute  to  optimal 
solutions.  This  hill  climbing  capability  is  the  basis  for  the  search  strategy's  name. 
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1.2  Examples 

Numerous  search  strategies  can  be  described  using  the  generalized  hill  climbing  algorithm 
framework.  Several  such  strategies  are  described  that  illustrate  the  breadth  of  algorithms  that  fell  within  the 
generalized  hill  climbing  algorithm  framework,  as  well  as  demonstrate  the  unifying  nature  of  the 
generalized  climbing  algorithm  paradigm.  All  these  search  strategy  variations  are  captured  purely  through  a 
change  in  the  definitions  of  the  hill  climbing  random  variable  and  the  neighborhood  structure. 

The  two  extreme  search  strategy  variations,  Monte  Carlo  search  and  local  search,  can  be 
formulated  as  generalized  hill  climbing  algorithms.  Monte  Carlo  search  can  be  formulated  as  a  generalized 
hill  climbing  algorithm  by  setting  RkO^k*®)  =  +°°>  ®k  ® '  ew(®k)>  w(®  k)  =  ^ •  Local 

search  can  be  formulated  as  a  generalized  hill  climbing  algorithm  by  setting  Rk(®  k>®  )  =  0,  ® k 

©'  e»(©k)- 

Simulated  annealing  can  be  formulated  as  a  generalized  hill  climbing  algorithm  by  setting 
Rk(©k,ro')  =  -tkln(u)  for  all  ©  k  e  Q ,  © '  €»(©k),  where  tk  is  a  temperature  parameter,  and  u  is  a 
U(0,1)  random  variable.  Note  that  Rk(©k,©')  is  an  exponential  random  variable  with  mean  tk  (from  the 
inversion  method  for  generating  exponential  random  variables).  Moreover,  for 
A(©k,©')=f(©')~  f(©k)£  0,  the  acceptance  probability  can  be  written  as 

P{Rk(©k,©')  ^f(©')-f(Ok)}=p{-tkln(u)^  f(©k)) 

=P{u  <  e-llf(a,')_f(®k)1/tk1}  =  e"llf(co,)"f(®k)]/tkl, 
which  is  how  simulated  annealing  is  commonly  described.  Therefore,  simulated  annealing  is  a  particular 
type  of  generalized  hill  climbing  algorithm. 

Threshold  accepting  can  be  formulated  as  a  generalized  hill  climbing  algorithm  by  setting 
Rk(©k,©')  =  Qk  for  all  cok  efl,  ffl'  €/j(cok)  where  Qk  is  a  non-negative  real  constant.  Therefore, 

for  A(©  k,©')=  f  (© ')  —  f(©  k  )  —  0,the  acceptance  probability  can  be  written  as 

P{Rk(©k,©')  >  f(©')  -f(fflk)}  =  P{Qk  ^  f(©0  -f(©k»  (!) 

which  is  how  threshold  accepting  is  commonly  described.  Therefore,  threshold  accepting  is  a  particular 
type  of  generalized  hill  climbing  algorithm. 

Ordinal  optimization  (Ho  1995)  is  a  search  strategy  that  focuses  on  finding  good  solutions,  rather 
than  finding  the  very  best  solution.  In  doing  this,  it  reduces  the  search  for  an  optimal  solution  from 
sampling  over  a  very  large  solution  space  to  sampling  over  a  smaller,  more  manageable,  set  of  good 
solutions.  The  ordinal  optimization  approach  can  be  incorporated  into  the  generalized  hill  climbing 
algorithm  framework.  The  resulting  algorithm,  termed  ordinal  hill  climbing  (OHC),  is  a  particular 
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generalized  hill  climbing  algorithm,  with  the  hill  climbing  component  bused  on  ordinal  information .  The 
OHC  algorithm  is  summarized  in  pseudo-code  form. 

Select  a  set  of  M  initial  solutions  D( 0)  c  fi 
Define  an  objective  function  f:  Q  — >  /? 

Define  a  neighborhood  structure  n :  Cl  ->  2n 
Set  the  iteration  index  k=0 

Define  the  hill  climbing  (random)  variables  Rk :  ->  Z+,  where 

0  Sm=l  P{Rk  =  m}  =  lforaUkeZ+ 
ii)  Ro  =1  with  probability  one. 

Hi)  Rk  — >  M  with  probability  one  as  k  — >  -H» 

Repeat 

Order  the  solutions  in  Z)(k)  from  smallest  to  largest  objective  function  values 
Generate  Rk 

Keep  the  best  (smallest  objective  function  values)  Rk  solutions  from  the  set  D( k). 

Call  this  set  Ei . 

Generate  one  or  more  neighbors  of  each  of  these  Rk ,  to  obtain  M-R  new  solutions. 

Call  this  set  E2 . 

Set  Z)(k+1)  <-E!uE2. 
k<—  k+1 

Until  stopping  criterion  is  met 

Tabu  search  with  short  term  memory  can  be  formulated  as  a  generalized  hill  climbing  algorithm  by 

setting 

Rk(<°t,o>  ')=["&'£■ =/*»'  *m  1  -«»'*«]-  w  emi-m'em  <2> 

for  all  (Ok  ef2\{(y'}  €«(©,.),  where  I  is  a  tabu  list  of  solutions  (Glover  1995)  and /{.}  is  a  0-1  indicator 
function. 

Erlang  accepting  can  be  formulated  as  a  generalized  hill  climbing  algorithm  by  setting 

Rk(©k,«>')=  -tk[ln(ui)+ln(u2)]  (3) 

for  all  ©  k  eCl,  <a'  e«(cok) ,  where  tk  is  a  temperature  parameter,  and  Ui  and  u2  are  independent  and 
identically  distributed  U(0,1)  random  variables.  The  resulting  generalized  hill  climbing  algorithm  is  called 
Erlang  accepting  since  Rk  is  distributed  Erlang  with  mean  2tk 
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Geometric  accepting  can  be  formulated  as  a  generalized  hill  climbing  algorithm  by  setting 

Rk(“k,<o')  =  rin(1-u)/ln(1_Pk)l  (4> 

for  all  ©  k  e  Q ,  © '  e  «(cok) ,  where  pk  6(0,1)  is  a  sampling  parameter,  u  is  a  U(0,1)  random  variable, 
and  H  denotes  the  ceiling  function.  The  resulting  generalized  hiU  climbing  algorithm  is  called  geometric 
accepting  since  Rk  is  distributed  geometric  with  mean  1/  pk  • 

Weibull  accepting  can  be  formulated  as  a  generalized  hill  climbing  algorithm  by  setting 
Rk(fflk,co')=  tk(-ln(u))1/<X  for  all  ©k  efl,  o'  e«(©k),  where  tk  is  a  temperature  parameter,  u  is  a 
U(0,1)  random  variable,  and  a  is  a  shape  parameter.  The  resulting  generalized  hill  climbing  algorithm  is 
called  Weibull  accepting  since  Rk  is  distributed  Weibull  with  mean  (t  k  /  a>r (1  /  a) . 

The  examples  presented  here  are  just  a  sampling  of  the  unlimited  number  of  new  (and  existing) 
search  strategies  that  can  be  created  (and  described)  using  the  generalized  hill  climbing  paradigm.  These 
examples  serve  to  illustrate  the  breadth,  scope,  and  variation  of  algorithms  that  lie  within  the  generalized 
hill  climbing  framework. 

1.3  Convergence  Results 

Convergence  conditions  provide  a  theoretical  understanding  of  generalized  hill  climbing  algorithms. 
Two  sufficient  conditions  for  convergence  of  generalized  hill  climbing  algorithms  were  derived.  The  first 
result  focuses  on  a  particular  class  of  generalized  hill  climbing  algorithms.  The  complete  details  of  this 
result  are  reported  in  Johnson  and  Jacobson  (1996)  as  well  as  the  Ph.D.  dissertation  of  Major  A.W. 
Johnson,  USAF. 

Theorem  1:  Given  a  discrete  optimization  problem,  with  solution  space  Cl,  objective  function  f,  and 
neighborhood  structure  n,  suppose  that  ©  *  denotes  a  global  optimum,  with  objective  function  value  f(to  *). 
Consider  the  class  of  generalized  hill  climbing  algorithms  with  hill  climbing  (random)  variable 
Rk((0k>(°,)>  ©k  sQ,  co'  e  nfak)  >  defined  with  acceptance  probability 

P{Rk(©k,©')^A(©k,©')=min[l,(P{Rk(©*,©')^A(©*,©')}/P{Rk(©*,©k)^  A(©*,©k)})L  (5) 
where  A(©  ,©  ")  =  f(©")-f(©)  and  Rk  is  any  arbitrary  non-negative  random  variable.  Then  the  resulting 

generalized  hill  climbing  algorithm  converges  in  probability  to  the  set  of  globally  optimal  solutions  if 

i)  P{Rk(<ak,©')^  A(cok,co')}>0  for  allk=0,l,..., 

ii)  P{Rk(©k,©')  ^  A(©k,©')}“*  0  as 

Theorem  1  provides  sufficient  convergence  conditions  for  a  particular  class  of  generalized  hill 
climbing  algorithms,  where  the  acceptance  probability  is  defined  by  (5).  This  class  of  algorithms  mclude, 
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for  example,  simulated  annealing.  A  large  body  of  convergent  generalized  hill  climbing  algorithms  can  be 
created  from  (5),  provided  /)  and  //)  are  satisfied. 

The  second  result  focuses  on  a  general  sufficient  convergence  condition  for  generalized  hill 
climbing  algorithms.  The  complete  details  of  this  result  are  reported  in  Johnson  and  Jacobson  (1997)  as 
well  as  the  Ph.D.  dissertation  of  Major  A.W.  Johnson,  USAF.  To  describe  this  result,  a  number  of 
definitions  are  needed. 

For  a  particular  discrete  optimization  problem,  with  a  solution  space,  neighborhood  structure,  and 
objective  function,  decompose  the  solution  space  into  the  set  of  global  optima,  Gcfl,  the  set  of  local 
optima,  LcQ,  and  all  other  solution  space  elements,  H=Q\(GuL).  Define  a  path  from  i  to  j  (i->j), 
ijeGuL,  as  a  sequence  of  solutions  A,oAq,...,A,<j  sf2,  with  A,q  =i,  Xjj  =j, 

X,r  €«(^r-l)  r=l,2,...,d.  Therefore,  paths  are  sequences  of  solutions  between  local  and  global  optima. 
Two  paths  from  i  to  j  are  equivalent  if  all  the  solutions  along  both  paths  are  identical,  and  the  order  in 
which  each  solution  is  visited  along  both  paths  is  identical.  Suppose  that  all  the  paths  from  i  to  j  are 
labelled,  hence  the  n*  path  from  i  to  j  is  denoted  by  i  — >n  j .  For  each  distinct  path  from  i  to  j,  the 

probability  of  the  n*  path  for  keZ+,  Pk(i  ->n  j),  is  the  product  of  the  probabilities  associated  with  each 

solution  transition.  Therefore,  for  k  eZ+,  the  probability  of  a  path  from  i  to  j  is 

Pk(i  — >  j)  =  Zn  Pk(i  ~*n  j)  f°r  a11  *>  J  eLuG,  i*j. 

If  i=j,  then 

Pk(i  — >  i)  =  1  —  2s€E(LuG)\{i}  Pk  0  “>  s) 

For  k  eZ+,  the  path  of  minimum  positive  probability  from  L  to  LvjG  is 
Pk (MinPath)  =  min{P|c (j-»i)|  j  eL,  ieLuG,  with Pk(j  — > i) >  0}. 

The  path  of  minimum  positive  probability  defines  the  path  with  the  smallest  probability  over  all  paths  from 
local  optima  to  local  or  global  optima.  Therefore,  this  probability  is  associated  with  the  most  difficult  path 
from  a  local  optima  to  a  local  or  global  optima. 

For  k  eZ+,  the  path  of  maximum  probability  from  G  to  Lis 

Pk  ( MaxPath )  =  max{Pk  (i  -» j)  |  i  e  G,  j  e  L} .  (9) 

The  path  of  maximum  probability  defines  the  path  with  the  largest  probability  over  all  paths  from  global 
optima  to  local  optima.  Therefore,  this  probability  is  associated  with  the  easiest  path  from  a  global  optima 

to  a  local  optima. 

For  k  eZ+  the  maximal  product  of  locally  optimal  solution  equilibrium  probabilities  and  their 
associated  path  probabilities  to  other  local  optima  is 


(6) 

(7) 

(8) 
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Vk(Max_Prod)=  max{5j(k)Pk(j  ->q)|  j,q  eL,  q*j,  q  tn(j)},  (10) 

where  5j(k)=  ^(kySieLuG^iC^)  and7ti(k),  i  eQ  are  the  equilibrium  probabilities  for  all  the 
solutions  in  Cl . 

Using  these  definitions.  Theorem  2  gives  sufficient  conditions  for  the  convergence  of  generalized 
hill  climbing  algorithms. 

Theorem  2:  Given  a  discrete  optimization  problem,  with  solution  space  Cl,  objective  function  f,  and 
neighborhood  structure  n,  if 

i)  Zk?i  Pk  (Min  Path )  =  +oo , 

//)  Zk?i  ¥k(Max_Path)  <  +°°, 
iii)  P ^(MaxProd)  <  +oo, 
then  ZieG  Jii(k)->  1  as  k— »-ko . 

Theorem  2  provides  sufficient  convergence  conditions  for  generalized  hill  climbing  algorithms,  by 
showing  that  if  conditions  /),  ii),  and  Hi)  hold,  then  the  equilibrium  probability  of  the  global  optima  will 
converge  to  one  as  k  approaches  infinity.  Informally,  these  conditions  state  that  the  probability  of  escaping 
from  a  local  optimum  must  approach  zero  slower  than  the  probability  of  escaping  from  a  global  optimum. 
These  conditions  are  based  on  the  path  construction  described  above.  These  conditions  are  the  least 
restrictive  convergence  conditions  available  in  the  literature  for  any  type  of  hill  climbing  algorithm. 

2.  Applications 

This  section  describes  applications  of  generalized  hill  climbing  algorithms  to  manufacturing 
process  design  optimization  problems  of  interest  to  the  Air  Force.  This  effort  has  been  interdisciplinary, 
involving  researchers  from  the  Materials  Process  Design  Branch  of  Wright  Laboratory  (Wright  Patterson 
Air  Force  Base),  Ohio  University,  and  Harvard  University. 

2.1  Discrete  Manufacturing  Process  Design  Optimization  Problems 

The  Materials  Process  Design  Branch  of  Wright  Laboratory  at  Wright  Patterson  Air  Force  Base 
(Dayton,  Ohio),  is  faced  with  the  challenge  of  identifying  robust  manufacturing  process  designs,  where  the 
finished  unit  meets  certain  geometric  and  microstructural  specifications,  and  is  produced  at  minimum  cost. 
To  date,  the  very  expensive  and  time  intensive  approach  of  trial  and  error  (on  the  shop  floor)  has  been  used 
to  identify  feasible  manufacturing  process  designs.  Wright  laboratory,  in  conjunction  with  researchers  at 
Ohio  University  (Athens,  Ohio),  have  developed  computer  simulation  models  of  manufacturing  processes, 
such  as  forging  and  machining.  Each  such  process  affects  the  geometry  and/or  microstructure  of  the 
manufactured  unit.  Associated  with  each  process  are  input  (controllable  and  uncontrollable)  and  output 


10 


parameters.  An  exhaustive  search  through  all  possible  process  sequences  and  controllable  input  parameters 
can  be  undertaken,  though  such  a  search,  especially  using  computer  simulation  models,  would  take  a 
prohibitive  amount  of  time,  hence  is  infeasible.  Therefore,  it  is  necessary  to  construct  efficient  and  effective 
optimization  algorithms  to  identify  optimal/near-optimal  designs  for  manufacturers  utilizing  manufacturing 
process  design  computer  simulation  models  in  their  manufacturing  process  design  planning  operations. 

To  describe  the  discrete  manufacturing  process  design  optimization  problem,  and  how  generalized 
hill  climbing  algorithms  can  be  used  to  address  the  problem,  a  number  of  definitions  are  needed.  Let  the 
manufacturing  processes  be  denoted  by  fy,P2,  ...,Pn .  Associated  with  each  process  are  (continuous  or 
discrete)  controllable  input  parameters,  uncontrollable  input  parameters,  and  output  parametersD.  The 
output  parameters  for  a  particular  process  may  serve  as  the  uncontrollable  input  parameters  for  a 
subsequent  process.  A  sequence  of  processes,  together  with  a  particular  set  of  input  parameters  constitutes 
a  manufacturing  process  design.  Label  such  designs  Dj,D2,...,D^j  .  Note  that  if  any  one  controllable 
input  parameter  is  continuous,  then  N  =  +°°  •  Otherwise,  N  <  +00  •  Without  loss  of  generality,  assume  that 
all  the  controllable  input  parameters  are  discrete  since  any  continuous  controllable  input  parameter  can  be 
discretized  over  an  arbitrarily  fine  grid.  Under  this  assumption,  the  number  of  manufacturing  process 
designs  is  finite,  though  potentially  very  large. 

Denote  the  design  space  by  O,  the  set  of  all  manufacturing  process  designs  (i.e., 
Q={D1,D2,...,Dn  }),  where  a  subset  of  these  designs  are  feasible.  Infeasible  designs  violate  prespecified 

constraints  on  the  manufacturing  process  and  the  unit  being  manufactured,  including  geometric  and 
microstructural  properties,  and  constraint  violations  on  the  output  parameters  (e.g.,  the  required  forging 
press  pressure  may  not  exceed  its  upper  bound  limitations).  Define  a  cost  function  f:  Q  -» [0,-t-oo)  that 
assigns  a  non-negative  value  to  each  element  of  the  design  space,  where  cost  includes  monetary  costs  as 
well  as  measures  for  how  well  the  finished  unit  meets  prespecified  geometric  and  microstructural 
properties,  penalties  for  constraint  violations  on  the  output  parameters,  and  measures  that  ensure  a  robust 
manufacturing  design  (i.e.,  the  manufacturing  process  is  stable)  (note  that  the  design  space  corresponds  to 
the  solution  space,  and  the  cost  function  corresponds  to  the  objective  function,  as  defined  in  Section  1). 
Define  a  neighborhood  structure  n  where  n(D)cQ  for  all  DeO.  The  neighborhood 

structure  establishes  connections  between  the  designs  in  the  design  space,  hence  allows  the  design  space  to 
be  traversed  or  searched  by  moving  between  designs  For  each  (fixed)  manufacturing  process  design 
sequence,  the  neighborhood  rule  allows  movement  between  controllable  input  parameter  values.  The  goal  is 
to  identify  the  globally  optimal  manufacturing  process  design  D*  (i.e.,  f(D*)  <  f(D)  for  all  D  eQ),  or 
more  realistically,  near-optimal  designs. 
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2.2  Integrated  Blade  Rotor 

The  Materials  Process  Design  Branch  of  Wright  Laboratory  identified  an  integrated  blade  rotor  as 
a  part  of  interest  to  the  Air  Force  for  developing  optimal  manufacturing  process  designs.  Therefore,  a 
manufacturing  process  design  is  needed  to  transform  a  billet  into  the  integrated  blade  rotor  geometric  shape 
(Gunasekera  et  al.  1996).  Three  manufacturing  process  sequences  have  been  identified  that  can  achieve  this 
transformation.  Define  the  following  notation  for  the  six  processes  that  make  up  these  three  designs: 

Pj  is  the  cast  ingot  process  P2  is  the  upset  process 

P3  is  the  machine  preform  process  P4  is  the  blocker  forge  process 

P5  is  the  rough  machining  process  P6  is  the  finished  shape  process 

The  three  possible  manufacturing  process  design  sequences,  provided  by  researchers  at  Wright 
Laboratory  and  Ohio  University,  are 

W5P6  W4P6  P1W4P6 

Associated  with  each  manufacturing  process  are  uncontrollable  and  controllable  input  parameters, 
and  its  output  parameters.  For  example,  for  process  Pj ,  there  are  two  controllable  input  parameters  (the 
radius  of  the  billet  and  the  height  of  the  billet),  zero  uncontrollable  input  parameters,  and  two  output 
parameters,  which  are  just  the  controllable  input  parameter  values.  See  Appendix  1  for  a  list  of  the 
controllable  input  parameters  for  each  of  the  six  processes.  Gunasekera  et  al.  1996  and  Fischer  et  al.  1997 
present  complete  details  on  all  six  processes  and  their  parameters. 

Associated  with  each  controllable  input  parameter  is  a  discrete  (naturally,  or  discretized  from  a 
continuous  domain)  set  of  feasible  values  (Appendix  1  contains  a  listing  of  these  values  for  all  the 
controllable  input  parameters  for  the  six  processes).  The  cost  function  quantifies  the  cost  associated  with 
not  meeting  certain  geometric  and  microstructural  properties  of  the  finished  product,  the  monetary  cost  in 
producing  the  finished  product,  and  cost  penalties  for  constraint  violations.  The  goal  is  to  identify  a 
manufacturing  process  sequence,  together  with  values  for  all  the  controllable  input  parameters  for  the 
processes,  that  result  in  a  feasible  manufacturing  process  design  that  produces  the  integrated  blade  rotor 
part  at  total  minimum  cost. 

Computer  simulation  models  of  the  manufacturing  processes  described  above  have  been  developed 
(Gunasekera  et  al.  1996,  Fischer  et  al.  1997).  This  moves  the  search  for  an  optimal  manufacturing  process 
design  from  the  shop  floor  (where  trial  and  error  has  typically  been  applied,  using  actual  materials)  to  a 
computer  platform.  However,  even  using  high  speed  computing  resources,  the  search  for  an  optimal 
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manufacturing  process  design  can  take  a  prohibitive  amount  of  time.  To  circumvent  this  problem, 
generalized  hill  climbing  algorithms  are  used  with  the  computer  simulation  models  to  identify  optimal/near- 
optimal  manufacturing  process  designs. 

Computational  results  for  the  integrated  blade  rotor  manufacturing  process  designs  are  reported 
with  GHC  algorithms  applied  to  the  manufacturing  process  computer  simulation  models.  The  complete 
details  of  these  experiments  are  available  in  Jacobson,  Sullivan  and  Johnson  (1997)  (note  that  this  paper 
received  the  best  paper  award  in  the  Industrial  Simulation  track  at  the  1997  European  Simulation 
Multiconference,  held  June  1-4,  1997,  in  Istanbul,  Turkey).  Five  different  GHC  formulations  (simulated 
annealing  threshold  accepting,  Monte  Carlo  search,  local  search,  and  Weibull  accepting),  together  with 
three  different  neighborhood  rules,  were  applied  to  the  three  computer  simulation  manufacturing  process 
design  sequences.  The  objective  in  running  these  experiments  is  to  assess  the  performance  of  these  GHC 
formulations  and  neighborhood  rules,  as  well  as  identify  optimal/near  optimal  manufacturing  process 
design  sequences  and  input  parameters  using  computer  simulation  models. 

For  both  simulated  annealing  and  Weibull  accepting,  t^  is  updated  by  multiplying  the  previous 

temperature  parameter  by  the  increment  multiplier,  Pj ,  where  0  <  Pi  <  1  (i.e.,  t^  =  Pit k—  l)  • 
temperature  parameter  is  t0  =  10,000,  with  p!  =  .96 .  For  Weibull  accepting,  the  shape  parameter  is  set 
to  a  =  2.0 .  For  threshold  accepting,  the  initial  threshold  is  Qo  =10,000,  with  Qk  =  p2Qk-l  ^  P2=-96- 
For  Monte  Carlo  search,  note  that  since  Rk(D,D')=  +  °°,  D,D'eQ,  for  all  k  eZ  then  all  uphill  moves 
are  accepted.  Conversely,  for  local  search,  since  Rk(D,D')=0,  D.D'efl,  for  all  k  eZ  then  all  uphill 
moves  are  rejected. 

Three  neighborhood  rules  were  implemented.  The  first  neighborhood  rule  consists  of  randomly 
selecting  one  process  in  a  given  design  sequence,  and  randomly  changing  (using  the  discrete  uniform  over 
the  set  of  possible  values,  as  listed  in  Appendix  1)  one  of  the  controllable  input  parameters  associated  with 
this  process.  The  second  neighborhood  rule  is  similar  to  the  first,  but  the  algorithm  randomly  changes  one 
controllable  input  parameter  in  each  of  the  processes  of  the  design  sequence  (rather  than  in  just  one 
particular  process).  Similar  to  the  second  neighborhood  rule,  the  third  neighborhood  rule  considers  every 
process  in  the  design  sequence.  However,  the  key  difference  is  that  each  controllable  input  parameter  in  all 

the  processes  is  changed  with  a  prespecified  probability  (p=.3). 

The  cost  function  evaluates  the  cost  associated  with  the  simulated  manufacturing  process  design,  in 
US  dollars.  The  initial  cost  is  the  cost  of  the  initial  billet,  that  depends  on  the  dimensions  of  the  billet  and 
the  specific  metal  being  processed.  The  costs  for  the  forging  processes  include: 
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/)  set  up  costs, 

//)  post-inspection  costs, 

iii)  die  wear  costs, 

iv)  press  run  costs,  and 

v)  the  cost  of  possible  strain-induced-porosity  damage  in  the  workpiece. 

Penalties  are  incurred  with  the  forging  processes  when 

i)  the  press  capacity  is  exceeded, 
ii)  the  aspect  ratio  of  the  workpiece  is  too  large,  and 
Hi)  the  geometry  of  the  workpiece  conflicts  with  the  die  geometry. 

The  cost  to  machine  the  workpiece  is  the  cost  of  the  material  removed  from  the  workpiece,  where  a 
penalty  cost  is  incurred  when  the  geometry  of  the  workpiece  conflicts  with  the  desired  final  geometry  of  the 
workpiece  after  margining  After  the  workpiece  is  processed,  a  mandatory  ultrasonic  non-destructive 
evaluation  cost  and,  if  necessary,  a  cost  of  heat  treatment  is  accrued.  In  addition,  the  final  microstructure  of 
the  workpiece  is  evaluated;  if  the  microstructure  violates  predetermined  specifications,  a  penalty  cost  is 
incurred.  All  penalties  are  translated  into  US  dollars  in  the  cost  function. 

Computational  results  with  the  GHC  algorithms  and  the  neighborhood  rules  are  reported  for  the 
three  manufacturing  process  design  sequences.  All  the  GHC  algorithms  were  executed  with  K=250  and 
M=40,  resulting  in  10,000  total  iterations.  Thirty  replications  of  each  GHC  formulation  were  made,  each 
initialized  with  a  different  initial  design  and  a  different  seed  for  the  random  number  generator.  Common 
initial  designs  were  used  across  the  five  GHC  formulations  (when  the  process  design  sequences  and 
neighborhood  rules  were  fixed),  for  all  thirty  replications.  For  the  first  replication,  a  feasible  controllable 
input  parameters  setting  was  used,  set  by  the  user.  The  initial  input  parameters  settings  for  the  remaining 
twenty-nine  replications  were  obtained  by  randomly  selecting  a  neighbor  of  the  first  replication's  feasible 
controllable  input  parameters  setting.  The  mean  (|i)  and  standard  deviation  (a),  as  well  as  the  minimum 
and  maximum  cost  function  values,  were  computed  from  these  thirty  replications.  All  computational 
experiments  were  executed  on  a  SUN  ULTRA-1  workstation  (128  Mb  RAM).  Each  set  of  thirty 
replications  took  approximately  30  CPU  minutes.  A  listing  of  the  computer  code  (in  C)  for  the  generalized 
hill  climbing  algorithms  used  to  run  these  experiments  is  available  from  the  principal  investigator. 
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Table  1 


GHC  Algorithm  Computational  Results  with  Neighborhood  Rule  One 


Sequence 

GHC  Formulation 

M. 

cr 

minimum 

maximum 

P1P2P5P6 

Simulated  Annealing 

2239.3 

250.8 

1927.0 

2487.3 

Threshold  Accepting 

2305.1 

238.7 

1932.1 

2518.7 

Monte  Carlo 

3056.9 

108.1 

2484.3 

3076.6 

Local  Search 

2474.2 

550.2 

1919.3 

3179.5 

Weibull  Accepting 

2224.5 

247.7 

1941.0 

2532.7 

P1P2P4P6 

Simulated  Annealing 

2285.3 

6.3 

2277.9 

2293.0 

Threshold  Accepting 

2286.9 

16.3 

2275.7 

2330.6 

Monte  Carlo 

2245.3 

0.0 

2245.3 

2245.3 

Local  Search 

2450.0 

286.9 

2285.4 

3321.9 

Weibull  Accepting 

2284.8 

8.3 

2277.9 

2312.4 

P1P2P3P4P6 

Simulated  Annealing 

2415.7 

208.0 

2245.5 

2824.6 

Threshold  Accepting 

2505.6 

239.7 

2245.5 

2827.7 

Monte  Carlo 

2353.2 

0.0 

2353.2 

2353.2 

Local  Search 

2685.4 

279.1 

2277.1 

2975.1 

Weibull  Accepting 

2498.6 

238.8 

2245.5 

2824.6 

Sequence 

P1P2P5P6 


W4P6 


W3P4P6 


Table  2 

GHC  Algorithm  Computational  Results  with  Neighborhood  Rule  Two 


GHC  Formulation 

It 

Simulated  Annealing 

1921.8 

Threshold  Accepting 

1921.7 

Monte  Carlo 

2395.7 

Local  Search 

2097.2 

Weibull  Accepting 

1920.3 

Simulated  Annealing 

2238.8 

Threshold  Accepting 

2240.4 

Monte  Carlo 

2250.0 

Local  Search 

2439.0 

Weibull  Accepting 

2238.6 

Simulated  Annealing 

2264.3 

Threshold  Accepting 

2282.8 

Monte  Carlo 

2874.0 

Local  Search 

2406.2 

Weibull  Accepting 

2263.7 

a 

minimum 

maximum 

7.0 

1919.3 

1957.4 

2.2 

1919.3 

1927.0 

85.3 

1943.9 

2411.3 

255.9 

1919.3 

2458.0 

2.8 

1919.3 

1932.4 

1.5 

2238.4 

2244.2 

3.9 

2238.4 

2350.0 

1.2 

2248.0 

2254.6 

2.5 

2238.4 

2248.1 

1.1 

2238.4 

2244.2 

19.1 

2245.5 

2282.7 

21.4 

2245.5 

2301.6 

0.0 

2874.0 

2874.0 

327.0 

2245.5 

3543.1 

21.9 

2245.5 

2308.6 
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Table  3 

GHC  Algorithm  Computational  Results  with  Neighborhood  Rule  Three 


Sequence 

GHC  Formulation 

4 

a 

minimum 

maximum 

P1P2P5P6 

Simulated  Annealing 

2145.1 

257.5 

1927.0 

2471.2 

Threshold  Accepting 

1966.0 

13.3 

1950.6 

1977.3 

Monte  Carlo 

2421.6 

0.0 

2421.6 

2421.6 

Local  Search 

2131.8 

308.5 

1919.3 

3158.0 

Weibull  Accepting 

2042.4 

202.3 

1927.0 

2473.2 

P1P2P4P6 

Simulated  Annealing 

2247.4 

6.8 

2238.4 

2254.9 

Threshold  Accepting 

2279.0 

12.9 

2248.1 

2285.7 

Monte  Carlo 

2298.9 

6.6 

2263.7 

2300.1 

Local  Search 

2275.6 

14.0 

2248.1 

2297.9 

Weibull  Accepting 

2250.3 

9.6 

2238.4 

2275.7 

P1P2P3P4P6 

Simulated  Annealing 

2291.3 

43.0 

2245.5 

2403.3 

Threshold  Accepting 

2311.3 

20.4 

2266.4 

2320.3 

Monte  Carlo 

2836.4 

0.0 

2836.4 

2836.4 

Local  Search 

2619.1 

461.8 

2245.5 

3838.1 

Weibull  Accepting 

2299.8 

99.0 

2245.5 

2786.2 

The  results  in  Tables  1-3  illustrate  the  performance  of  the  five  GHC  algorithms  and  three 
neighborhood  rules  on  the  three  manufacturing  process  design  sequences.  In  particular,  simulated 
annealing,  threshold  accepting,  and  WeibuU  accepting  all  yielded  comparable  results  that  were  superior  to 
the  results  produced  by  Monte  Carlo  search  and  local  search.  Local  search  also  tended  to  yield  results  with 
higher  variance. 

Comparing  the  three  neighborhood  rules,  the  results  with  the  second  rule  were  superior  to  the 
results  with  the  first  and  third  rules.  The  first  rule  is  very  myopic  (conservative)  in  how  it  traverses  the 
design  space,  while  the  third  rule  is  more  aggressive;  the  second  rule  provides  an  effective  balance  between 
the  two. 

Comparing  the  three  design  sequences,  the  first  design  sequence  (P^Ps^)  yieWed  the  minimum 
cost  to  produce  the  integrated  blade  rotor  part.  This  is  consistent  with  manufacturing  process  practices, 
since  machining  is  more  cost  effective  if  the  material  being  worked  is  sufficiently  inexpensive  to  purchase 
and  easy  to  machine  (i.e.,  soft  versus  hard).  Overall,  the  computational  results  are  consistent  with  what 
would  have  been  obtained  using  trial  and  error  on  the  shop  floor.  The  advantage  of  using  generalized  hill 
climbing  algorithms  and  computer  simulation  manufacturing  processes  is  the  speed  and  efficiency  at  which 
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these  results  can  be  obtained,  at  a  fraction  of  the  cost  that  would  be  spent  if  trial  and  error  on  the  shop  floor 
would  be  required.  The  computational  results  reported  suggest  that  generalized  hill  climbing  algorithms 
provide  a  practical  tool  for  determining  optimal/near-optimal  manufacturing  process  designs  for  the 
integrated  blade  rotor  part.  The  application  of  optimization  algorithm  to  identify  good  manufacturing 
process  designs  using  computer  simulation  is  a  new  and  significant  advance  in  how  designs  can  be  cost- 
effectively  identified.  Work  is  in  progress  to  apply  this  approach  to  identify  good  manufacturing  process 
designs  for  other  complex  parts  of  interest  and  concern  to  the  Air  Force.  The  significant  savings  in  both 
time  and  cost  for  identifying  optimal  manufacturing  process  designs  has  the  potential  to  make  this  approach 
a  valuable  shop  floor  tool  for  real-time  analysis.  In  addition,  work  is  in  progress  to  transfer  this  technology 
into  the  hands  of  manufacturing  companies  who  can  benefit  from  its  application.  Several  manufacturing 
companies  (Pratt  &  Whitney,  Hughes  Research  Laboratories,  Alcoa,  and  Wyman-Gordon)  have  expressed 
an  interest  in  using  optimization  algorithms  to  address  their  computer  simulation  discrete  manufacturing 
process  design  optimization  problems.  These  companies  are  open  to  using  algorithms  that  have  the 
potential  to  globally  optimize  manufacturing  process  designs;  generalized  hill  climbing  algorithms  can 
provide  tools  to  fill  this  need.  Dr.  James  C.  Malas  at  the  Materials  Process  Design  Branch  of  Wright 
Laboratory  (Wright  Patterson  Air  Force  Base)  has  been  the  point  of  contact  with  these  companies. 

3.  Other  Research  Results 

In  addition  to  the  results  reported  with  generalized  hill  climbing  algorithms,  and  discrete 
manufacturing  process  design  optimization  problems,  several  other  results  were  obtained  during  the  period 
of  the  grant.  These  results  are  briefly  discussed  here. 

Discrete  optimization  and  discrete  event  simulation  are  two  disparate  areas  within  the  general  field 
of  operations  research.  Building  a  framework  that  facilitates  the  cross-fertilization  of  tools  in  these  two 
areas  has  provided  much  of  the  motivation  and  basic  research  leading  up  to  the  creation  of  the  generalized 
hill  climbing  algorithm  paradigm.  Seven  search  problem  for  discrete  event  simulation  models  have  been 
formulated  and  proven  to  be  NP-hard.  These  problems  consider  fundamental  issues  faced  by  simulation 
practitioners  (e.g.,  can  a  state  be  accessed,  does  changing  the  order  of  events  impact  the  states,  does 
representing  the  same  simulation  model  on  a  computer  in  two  different  ways  result  in  different  states  when 
executed,  can  a  simulation  run  stall,  how  do  permutations  of  events  impact  the  states  and  future  events 
lists,  or  how  events  are  cancelled,  resulting  in  a  loss  of  information).  The  first  problem,  termed 
ACCESSIBILITY,  is  a  generalization  of  the  discrete  manufacturing  design  problems  described  in  Section 
2.  Jacobson  and  Yucesan  (1995,  1998)  prove  these  problems  to  be  NP-hard,  and  discuss  their  practical 
implications  for  simulation  practitioners. 
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Fleischer  and  Jacobson  (1996,  1997)  consider  how  information  theory  can  be  used  to  assess  the 
finite-time  performance  of  the  simulated  annealing  algorithm.  These  results  provide  insights  into  how 
simulated  annealing  can  be  executed  to  optimize  its  effectiveness.  Jacobson  and  Schruben  (1998)  look  at 
how  harmonic  analysis  can  be  used  to  perform  sensitivity  analysis  of  steady  state  discrete  event  simulation 
models.  Kumar  et  al.  (1997)  identify  a  design  and  scheduling  optimization  problem  associated  with  flexible 
assembly  systems,  prove  the  problem  to  be  NP-hard,  identify  a  lower  bound  for  the  problem,  and  introduce 
an  efficient  deterministic  heuristic  to  address  the  problem.  Jacobson  et  al.  (1997)  also  report  on  the 
effectiveness  of  simulated  annealing  and  tabu  search  for  this  problem,  and  give  a  polynomial-time 
algorithm  to  compute  the  lower  bound.  Kobza  and  Jacobson  (1996,  1997)  provide  a  probabilistic  analysis 
of  access  control  security  systems.  They  consider  various  security  models,  and  show  how  design  of  such 
systems  impact  the  performance  of  the  security  systems.  Jacobson  and  Morrice  (1996,  1998)  study  a 
combinatorial  medical  problem  that  evaluates  the  temporal  association  between  health  disorders  and 
medical  treatments. 
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APPENDIX  1:  Input  Parameters  for  the  Integrated  Blade  Rotor 

The  following  are  the  input  parameters  for  the  six  processes: 


Process 


Input  Parameters 


Possible  Values 


Pi 

P2 


P3 


Billet  Radius 
Billet  Height 

Die  Geometry: 

Height  1 
Height  2 
Height  3 
Die  Speed 
Die  Friction  Factor 

Die  Temperature/Ambient  Temperature 

Radius 

Height 


[2,2.5,3.0,3.5,4.0,4.5,5.0,5.5,6.0] 
[1.0,2.0,3.0]*  Billet  Radius 


[1.5,  2.0,  2.5,  3.0,3. 5] 
[1.5,  2.0,  2.5,  3. 0,3.5] 
[1.5,  2.0,  2.5,  3.0,3.5] 
[.4,6,8,10,1-2] 
[,2,.4,.6,.8] 

,1607,1652,1697,1742,1787,1832 

[2.0,2.25,. ..,5.0] 
[1.5,2. 0,2.5, 3. 0,3. 5] 


P4 


P5 


P6 


Die  Geometry: 

Radius  1 
Height  1 
Radius  2 
Height  2 
Die  Speed 
Die  Friction  Factor 

Die  Temperature/Ambient  Temperature 


[1.5,  1.75,...,  3.5] 
[0.5,  0.625,  0.75,  0.875] 
9.0  (fixed) 

[1.0,1.25,1.5,1.75,2.0] 

[0.4,0.6,0.8,1.0,1.2] 

[0.2,0.4,0.6,0.8] 

[1562, 1607,1652, 1697, 1742, 1787, 1832] 


Radius  1 

[1.25, 1.50, ...,  3.25] 

Height  1 

[0.5,0.625,0.75,0.875] 

Radius  2 

[1.5,  1.75, ...,  3.5] 

Height  2 

[0.5,0.75,1.0,1.25,1.5,1.75,2.0] 

Radius  3 

[3.5,  3.75, ...,  5.0] 

Height  3 

[1.0,1.25,1.5,1.75,2.0] 

Radius  1 

3.0  (fixed) 

Height  1 

0.5  (fixed) 

Radius  2 

3.0  (fixed) 

Height  2 

1.5  (fixed) 

Radius  3 

4.0  (fixed) 

Height  3 

1.5  (fixed) 
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